Neural Information Processing Systems http://nips.cc/

We quantify the efficiency of temporal difference (TD) learning over the direct, or Monte Carlo (MC), estimator for policy evaluation in reinforcement learning, with an emphasis on estimation of quantities related to rare events. Policy evaluation is complicated in the rare event setting by the long timescale of the event and by the need for \emph{relative accuracy} in estimates of very small values. Specifically, we focus on least-squares TD (LSTD) prediction for finite state Markov chains, and show that LSTD can achieve relative accuracy far more efficiently than MC. We prove a central limit theorem for the LSTD estimator and upper bound the \emph{relative asymptotic variance} by simple quantities characterizing the connectivity of states relative to the transition probabilities between them. Using this bound, we show that, even when both the timescale of the rare event and the relative accuracy of the MC estimator are exponentially large in the number of states, LSTD maintains a fixed level of relative accuracy with a total number of observed transitions of the Markov chain that is only \emph{polynomially} large in the number of states.

What is being transferred in transfer learning?

Self-attention mechanisms are foundational to Transformer architectures, supporting their impressive success in a wide range of tasks. While there are many self-attention variants, their robustness to noise and spurious correlations has not been well studied. This study evaluates Softmax, Sigmoid, Linear, Doubly Stochastic, and Cosine attention within Vision Transformers under different data corruption scenarios. Through testing across the CIFAR-10, CIFAR-100, and Imagenette datasets, we show that Doubly Stochastic attention is the most robust. It consistently outperformed the next best mechanism by $0.1\%-5.1\%$ when training data, or both training and testing data, were corrupted. Our findings inform self-attention selection in contexts with imperfect data. The code used is available at https://github.com/ctamayor/NeurIPS-Robustness-ViT.

From infancy, humans have expectations about how objects will move and interact.

We quantify the efficiency of temporal difference (TD) learning over the direct, or Monte Carlo (MC), estimator for policy evaluation in reinforcement learning, with an emphasis on estimation of quantities related to rare events. Policy evaluation is complicated in the rare event setting by the long timescale of the event and by the need for \emph{relative accuracy} in estimates of very small values. Specifically, we focus on least-squares TD (LSTD) prediction for finite state Markov chains, and show that LSTD can achieve relative accuracy far more efficiently than MC. We prove a central limit theorem for the LSTD estimator and upper bound the \emph{relative asymptotic variance} by simple quantities characterizing the connectivity of states relative to the transition probabilities between them. Using this bound, we show that, even when both the timescale of the rare event and the relative accuracy of the MC estimator are exponentially large in the number of states, LSTD maintains a fixed level of relative accuracy with a total number of observed transitions of the Markov chain that is only \emph{polynomially} large in the number of states.

Data valuation, which measures the contribution of individual data source on machine learning (ML) model performance, plays a crucial role in improving algorithmic transparency and creating incentive mechanisms for data sharing and monetization (Liu et al., 2023). Its importance is particularly evident in sectors like healthcare and finance, where explainable ML is increasingly being adopted for high-stake decision-making (Sahoh and Choksuriwong, 2023). The recent rise of data marketplaces further highlights the need for accurate data valuation (Ghorbani and Zou, 2019; Jia et al., 2019a). By integrating diverse data sources, these marketplaces enhance ML tasks and unlock significant business values (Agarwal et al., 2019). Fair compensation for data creators based on the value of their data is crucial in such contexts, making the equitable valuation of data a key issue (Altman, 2023). Data Shapley has recently gained widespread recognition for quantifying the contribution of individual data points to ML models (Ghorbani and Zou, 2019; Jia et al., 2019b). It is uniquely defined by four axioms (see Axiom 2.1-2.4 in Section 2).